Clustering words to match conditions: an algorithm for stimuli selection in factorial designs / Agrupar palabras para igualar condiciones: un algoritmo para la selección de palabras en diseños factoriales

With the increasing refinement of language processing models and the new discoveries about which variables can modulate these processes, stimuli selection for experiments with a factorial design is becoming a tough task. Selecting sets of words that differ in one variable, while matching these same words into dozens of other confounding variables is time consuming and error prone. To assist experimenters in this thankless task, we present a simple method to perform it with little effort. The method is based on Kmeans clustering as a way to detect small and tight clusters of words that match in the desired variables. We have formalized the procedure into an algorithmic format, that is, a series of easy-to-follow steps. In addition, we also provide an SPSS syntax that helps in choosing the correct size of the clustering. After reviewing the theory, we present a worked example that will guide the reader through the complete procedure. The dataset of the worked example is available as a supplementary material to this paper (AU)

Con el creciente refinamiento de los modelos de procesamiento del lenguaje y los nuevos hallazgos sobre qué variables pueden modular dichos procesos, la selección de palabras para experimentos de diseño factorial se está convirtiendo en una tarea cada vez más ardua. Seleccionar conjuntos de palabras que difieren en una variable pero que están igualadas en una decena de posibles variables extrañas, lleva mucho tiempo y está sujeto a errores. Para ayudar a los experimentadores en esta desagradecida tarea, presentamos un método sencillo que permite realizarla con poco esfuerzo. El método se basa en el agrupamiento de Kmedias para identificar conjuntos pequeños y compactos de palabras igualadas en las variables deseadas. El procedimiento ha sido formalizado en un algoritmo, esto es, una serie de pasos concretos y sencillos de seguir. Además, también aportamos la sintaxis en SPSS para ayudar en la selección del número adecuado de agrupaciones. Tras una revisión de la teoría, presentamos un ejemplo práctico que guiará al lector a través del procedimiento completo. El conjunto de datos del ejemplo se encuentra disponible como material complementario a este artículo (AU)

Financial surrogate decision making: lessons from applied experimental philosophy

An estimated 1 in 4 elderly Americans need a surrogate to make decisions at least once in their lives. With an aging population, that number is almost certainly going to increase. This paper focuses on financial surrogate decision making. To illustrate some of the empirical and moral implications associated with financial surrogate decision making, two experiments suggest that default choice settings can predictably influence some surrogate financial decision making. Experiment 1 suggested that when making hypothetical financial decisions, surrogates tended to stay with default settings (OR = 4.37, 95% CI 1.52, 12.48). Experiment 2 replicated and extended this finding suggesting that in a different context (OR = 2.27, 95% CI 1.1, 4.65). Experiment 2 also suggested that those who were more numerate were less likely to be influenced by default settings than the less numerate, but only when the decision is whether to «opt in» (p = .05). These data highlight the importance of a recent debate about «nudging». Defaults are common methods to nudge people to make desirable choices while allowing the liberty to choose otherwise. Some of the ethics of using default settings to nudge surrogate decision makers are discussed (AU)

No dispoinble

Discriminaciones condicionales con estímulos contextuales versus complejos: implicaciones en la transferencia del aprendizaje / Conditional discriminations using contextual versus complex stimuli: implication for learning transfer

No disponible

This study sought to determine the effect of two conditional discrimination procedures involving contextual vs complex stimuli, on two types of learning transfer. For this purpose, three experiments were carried out. The first experiment used a contextual control procedure and the second a complex control procedure. The third experiment compared the two procedures using an inter-group design. The first two experiments aimed to determine the effect of the teaching procedure used on the transfer of learning to new stimulus relations (first type of transfer). The third experiment sought to determine the effect of teaching conditional discriminations using wither contextual or complex stimuli on the other discrimination (second type of transfer). In the first experiment, four out of six subjects displayed transfer of learning to new stimulus relations; in the second, only two subjects displayed that transfer. In the third experiment, statistically-significant differences were recorded for the dependent variable: successful attempts in the test using contextual stimuli. The teaching of tasks using contextual stimuli appears to favour the transfer of learning to conditional discriminations involving complex stimuli; the reverse, however, is not true to the same extent. These findings may be useful for the teaching of complex discriminations to intellectually-disabled or autistic children (AU)

The Search for Significance: A Few Peculiarities in the Distribution of P Values in Experimental Psychology Literature.

In this project I investigate the use and possible misuse of p values in papers published in five (high-ranked) journals in experimental psychology. I use a data set of over 135'000 p values from more than five thousand papers. I inspect (1) the way in which the p values are reported and (2) their distribution. The main findings are following: first, it appears that some authors choose the mode of reporting their results in an arbitrary way. Moreover, they often end up doing it in such a way that makes their findings seem more statistically significant than they really are (which is well known to improve the chances for publication). Specifically, they frequently report p values "just above" significance thresholds directly, whereas other values are reported by means of inequalities (e.g. "p<.1"), they round the p values down more eagerly than up and appear to choose between the significance thresholds and between one- and two-sided tests only after seeing the data. Further, about 9.2% of reported p values are inconsistent with their underlying statistics (e.g. F or t) and it appears that there are "too many" "just significant" values. One interpretation of this is that researchers tend to choose the model or include/discard observations to bring the p value to the right side of the threshold.

Comment on Hoffman and Rovine (2007): SPSS MIXED can estimate models with heterogeneous variances.

Hoffman and Rovine (Behavior Research Methods, 39:101-117, 2007) have provided a very nice overview of how multilevel models can be useful to experimental psychologists. They included two illustrative examples and provided both SAS and SPSS commands for estimating the models they reported. However, upon examining the SPSS syntax for the models reported in their Table 3, we found no syntax for models 2B and 3B, both of which have heterogeneous error variances. Instead, there is syntax that estimates similar models with homogeneous error variances and a comment stating that SPSS does not allow heterogeneous errors. But that is not correct. We provide SPSS MIXED commands to estimate models 2B and 3B with heterogeneous error variances and obtain results nearly identical to those reported by Hoffman and Rovine in their Table 3. Therefore, contrary to the comment in Hoffman and Rovine's syntax file, SPSS MIXED can estimate models with heterogeneous error variances.

Properties of hypothesis testing techniques and (Bayesian) model selection for exploration-based and theory-based (order-restricted) hypotheses.

In this paper, the performance of six types of techniques for comparisons of means is examined. These six emerge from the distinction between the method employed (hypothesis testing, model selection using information criteria, or Bayesian model selection) and the set of hypotheses that is investigated (a classical, exploration-based set of hypotheses containing equality constraints on the means, or a theory-based limited set of hypotheses with equality and/or order restrictions). A simulation study is conducted to examine the performance of these techniques. We demonstrate that, if one has specific, a priori specified hypotheses, confirmation (i.e., investigating theory-based hypotheses) has advantages over exploration (i.e., examining all possible equality-constrained hypotheses). Furthermore, examining reasonable order-restricted hypotheses has more power to detect the true effect/non-null hypothesis than evaluating only equality restrictions. Additionally, when investigating more than one theory-based hypothesis, model selection is preferred over hypothesis testing. Because of the first two results, we further examine the techniques that are able to evaluate order restrictions in a confirmatory fashion by examining their performance when the homogeneity of variance assumption is violated. Results show that the techniques are robust to heterogeneity when the sample sizes are equal. When the sample sizes are unequal, the performance is affected by heterogeneity. The size and direction of the deviations from the baseline, where there is no heterogeneity, depend on the effect size (of the means) and on the trend in the group variances with respect to the ordering of the group sizes. Importantly, the deviations are less pronounced when the group variances and sizes exhibit the same trend (e.g., are both increasing with group number).

Power and sample size calculations for testing linear combinations of group means under variance heterogeneity with applications to meta and moderation analyses

The general formulation of a linear combination of population means permits a wide range of research questions to be tested within the context of ANOVA. However, it has been stressed in many research areas that the homogeneous variances assumption is frequently violated. To accommodate the heterogeneity of variance structure, the Welch–Satterthwaite procedure is commonly used as an alternative to the t test for detecting the substantive significance of a linear combination of mean effects. This article presents two approaches to power and sample size calculations for the Welch–Satterthwaite test. The usefulness and diversity of the suggested techniques are illustrated with two of the potential applications in meta and moderation analyses. The numerical assessments showed that the proposed approaches outperform the existing methods on the accuracy of power calculations and sample size determinations for meta and moderation studies. Computer algorithms are also developed to implement the recommended procedures in actual research designs (AU)

No disponible

A note on Cohen's overlapping proportions of normal distributions.

Social scientists are often interested in computing the proportion of overlap and nonoverlap between two normal distributions that are separated by some magnitude. In his popular book, Statistical Power Analysis for the Behavioral Sciences (1988, 2nd ed.), Jacob Cohen provided a table (Table 2.2.1) for determining such proportions from common values of separation. Unfortunately, Cohen's proportions are inconsistent with his explication of the popular index of effect size, d; and his proportions are underestimates of distributional overlap and overestimates of nonoverlap. The authors explain how Cohen derived his values and then provide a revised, corrected table of proportions that also match values presented elsewhere.

Buy three but get only two: the smallest effect in a 2 × 2 ANOVA is always uninterpretable.

Loftus (Memory & Cognition 6:312-319, 1978) distinguished between interpretable and uninterpretable interactions. Uninterpretable interactions are ambiguous, because they may be due to two additive main effects (no interaction) and a nonlinear relationship between the (latent) outcome variable and its indicator. Interpretable interactions can only be due to the presence of a true interactive effect in the outcome variable, regardless of the relationship that it establishes with its indicator. In the present article, we first show that same problem can arise when an unmeasured mediator has a nonlinear effect on the measured outcome variable. Then we integrate Loftus's arguments with a seemingly contradictory approach to interactions suggested by Rosnow and Rosenthal (Psychological Bulletin 105:143-146, 1989). We show that entire data patterns, not just interaction effects alone, produce interpretable or noninterpretable interactions. Next, we show that the same problem of interpretability can apply to main effects. Lastly, we give concrete advice on what researchers can do to generate data patterns that provide unambiguous evidence for hypothesized interactions.

Analysis of the interaction between experimental and applied behavior analysis.

To study the influences between basic and applied research in behavior analysis, we analyzed the coauthorship interactions of authors who published in JABA and JEAB from 1980 to 2010. We paid particular attention to authors who published in both JABA and JEAB (dual authors) as potential agents of cross-field interactions. We present a comprehensive analysis of dual authors' coauthorship interactions using social networks methodology and key word analysis. The number of dual authors more than doubled (26 to 67) and their productivity tripled (7% to 26% of JABA and JEAB articles) between 1980 and 2010. Dual authors stood out in terms of number of collaborators, number of publications, and ability to interact with multiple groups within the field. The steady increase in JEAB and JABA interactions through coauthors and the increasing range of topics covered by dual authors provide a basis for optimism regarding the progressive integration of basic and applied behavior analysis.

The frequency of excess success for articles in Psychological Science.

Recent controversies have questioned the quality of scientific practice in the field of psychology, but these concerns are often based on anecdotes and seemingly isolated cases. To gain a broader perspective, this article applies an objective test for excess success to a large set of articles published in the journal Psychological Science between 2009 and 2012. When empirical studies succeed at a rate much higher than is appropriate for the estimated effects and sample sizes, readers should suspect that unsuccessful findings have been suppressed, the experiments or analyses were improper, or the theory does not properly account for the data. In total, problems appeared for 82 % (36 out of 44) of the articles in Psychological Science that had four or more experiments and could be analyzed.

Should we rely on the Kenward-Roger approximation when using linear mixed models if the groups have different distributions?

The study explores the robustness to violations of normality and sphericity of linear mixed models when they are used with the Kenward-Roger procedure (KR) in split-plot designs in which the groups have different distributions and sample sizes are small. The focus is on examining the effect of skewness and kurtosis. To this end, a Monte Carlo simulation study was carried out, involving a split-plot design with three levels of the between-subjects grouping factor and four levels of the within-subjects factor. The results show that: (1) the violation of the sphericity assumption did not affect KR robustness when the assumption of normality was not fulfilled; (2) the robustness of the KR procedure decreased as skewness in the distributions increased, there being no strong effect of kurtosis; and (3) the type of pairing between kurtosis and group size was shown to be a relevant variable to consider when using this procedure, especially when pairing is positive (i.e., when the largest group is associated with the largest value of the kurtosis coefficient and the smallest group with its smallest value). The KR procedure can be a good option for analysing repeated-measures data when the groups have different distributions, provided the total sample sizes are 45 or larger and the data are not highly or extremely skewed.

On the asymptotic standard error of a class of robust estimators of ability in dichotomous item response models.

In item response theory, the classical estimators of ability are highly sensitive to response disturbances and can return strongly biased estimates of the true underlying ability level. Robust methods were introduced to lessen the impact of such aberrant responses on the estimation process. The computation of asymptotic (i.e., large-sample) standard errors (ASE) for these robust estimators, however, has not yet been fully considered. This paper focuses on a broad class of robust ability estimators, defined by an appropriate selection of the weight function and the residual measure, for which the ASE is derived from the theory of estimating equations. The maximum likelihood (ML) and the robust estimators, together with their estimated ASEs, are then compared in a simulation study by generating random guessing disturbances. It is concluded that both the estimators and their ASE perform similarly in the absence of random guessing, while the robust estimator and its estimated ASE are less biased and outperform their ML counterparts in the presence of random guessing with large impact on the item response process.

An accumulator model for responses and response times in tests based on the proportional hazards model.

Latent trait models for responses and response times in tests often lack a substantial interpretation in terms of a cognitive process model. This is a drawback because process models are helpful in clarifying the meaning of the latent traits. In the present paper, a new model for responses and response times in tests is presented. The model is based on the proportional hazards model for competing risks. Two processes are assumed, one reflecting the increase in knowledge and the second the tendency to discontinue. The processes can be characterized by two proportional hazards models whose baseline hazard functions correspond to the temporary increase in knowledge and discouragement. The model can be calibrated with marginal maximum likelihood estimation and an application of the ECM algorithm. Two tests of model fit are proposed. The amenability of the proposed approaches to model calibration and model evaluation is demonstrated in a simulation study. Finally, the model is used for the analysis of two empirical data sets.

Model selection for minimum-diameter partitioning.

The minimum-diameter partitioning problem (MDPP) seeks to produce compact clusters, as measured by an overall goodness-of-fit measure known as the partition diameter, which represents the maximum dissimilarity between any two objects placed in the same cluster. Complete-linkage hierarchical clustering is perhaps the best-known heuristic method for the MDPP and has an extensive history of applications in psychological research. Unfortunately, this method has several inherent shortcomings that impede the model selection process, such as: (1) sensitivity to the input order of the objects, (2) failure to obtain a globally optimal minimum-diameter partition when cutting the tree at K clusters, and (3) the propensity for a large number of alternative minimum-diameter partitions for a given K. We propose that each of these problems can be addressed by applying an algorithm that finds all of the minimum-diameter partitions for different values of K. Model selection is then facilitated by considering, for each value of K, the reduction in the partition diameter, the number of alternative optima, and the partition agreement among the alternative optima. Using five examples from the empirical literature, we show the practical value of the proposed process for facilitating model selection for the MDPP.

Statistical mediation analysis with a multicategorical independent variable.

Virtually all discussions and applications of statistical mediation analysis have been based on the condition that the independent variable is dichotomous or continuous, even though investigators frequently are interested in testing mediation hypotheses involving a multicategorical independent variable (such as two or more experimental conditions relative to a control group). We provide a tutorial illustrating an approach to estimation of and inference about direct, indirect, and total effects in statistical mediation analysis with a multicategorical independent variable. The approach is mathematically equivalent to analysis of (co)variance and reproduces the observed and adjusted group means while also generating effects having simple interpretations. Supplementary material available online includes extensions to this approach and Mplus, SPSS, and SAS code that implements it.

Statistical power of latent growth curve models to detect quadratic growth.

Latent curve models (LCMs) have been used extensively to analyze longitudinal data. However, little is known about the power of LCMs to detect nonlinear trends when they are present in the data. For this study, we utilized simulated data to investigate the power of LCMs to detect the mean of the quadratic slope, Type I error rates, and rates of nonconvergence during the estimation of quadratic LCMs. Five factors were examined: the number of time points, growth magnitude, interindividual variability, sample size, and the R (2)s of the measured variables. The results showed that the empirical Type I error rates were close to the nominal value of 5 %. The empirical power to detect the mean of the quadratic slope was affected by the simulation factors. Finally, a substantial proportion of samples failed to converge under conditions of no to small variation in the quadratic factor, small sample sizes, and small R (2) of the repeated measures. In general, we recommended that quadratic LCMs be based on samples of (a) at least 250 but ideally 400, when four measurement points are available; (b) at least 100 but ideally 150, when six measurement points are available; (c) at least 50 but ideally 100, when ten measurement points are available.

Introduction to December 2013 issue.

In this introduction to the December 2013 issue of the Journal of Experimental Psychology: Applied, the editor discusses her goals to get the Journal back on track. She gives thanks for the research that continues to advance both science and practice in experimental psychology.